{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "U1VvuYAFIe6c"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Q2ahc5GhIe6n"
   },
   "source": [
    "### All the parameters required for the network can be tweaked below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GUDIDhX4Ie6o"
   },
   "outputs": [],
   "source": [
    "### Hyperparamets, change accordingly\n",
    "Channel_uses = 4\n",
    "Epochs = 2500\n",
    "Noise_variance = 1e-4\n",
    "Pert_variance = 1e-4\n",
    "Batch_size = 1024\n",
    "# init_losses_vec = np.ones(128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yRtqv5SCIe6r"
   },
   "source": [
    "### TX loss function \n",
    "The policy function for the transmitter is similar to that of a cross-entropy between the noisy loss feedback (l) and the J(w,$\\theta$) function value\n",
    "\n",
    "Loss = -$\\sum_{i=1}^n$($l_i$ * J($w_i$,$\\theta$))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_rajnM9xIe6r"
   },
   "outputs": [],
   "source": [
    "def tx_loss(y_true, y_pred): \n",
    "#     loss = - y_true*keras.backend.log(y_pred)\n",
    "\n",
    "    return -y_true*y_pred\n",
    "     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "k0Q5d2B-Ie6u"
   },
   "source": [
    "### Perturbation\n",
    "\n",
    "After we get the output from the transmitter network, we then add the perturbation matrix as mentioned in the paper. We write a  function for this purpose and then make a custom layer like functionality using the `keras.layers.lambda` functionality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-CtrC_BVIe6v"
   },
   "outputs": [],
   "source": [
    "def perturbation(d):\n",
    "    W = tf.keras.backend.random_normal(shape = (2*Channel_uses,),\n",
    "    mean=0.0,stddev=Pert_variance**0.5,dtype=None,seed=None)\n",
    "    d = ((1-Pert_variance)**0.5)*d + W\n",
    "    return d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uV6iehRhIe6x"
   },
   "source": [
    "###  Tx model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VAL-LujlIe6y"
   },
   "outputs": [],
   "source": [
    "def Int_layer(y):\n",
    "    w = y[:,y.shape[-1]//2:] - y[:,:y.shape[-1]//2]\n",
    "    print(w.shape)\n",
    "    t = -keras.backend.sum(w*w)\n",
    "#     t = keras.backend.exp(-t/Pert_variance**2)/(np.pi*Pert_variance**2)**Channel_uses\n",
    "    return t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 132
    },
    "colab_type": "code",
    "id": "bu0_LvHCIe60",
    "outputId": "8b908513-fa31-4e59-9831-19e5dd6aac9a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n",
      "hehehe (?, 8)\n",
      "(?, 16)\n"
     ]
    }
   ],
   "source": [
    "# tx layers\n",
    "tx_input = keras.layers.Input((1,), name='tx_input')\n",
    "x = keras.layers.BatchNormalization()(tx_input)\n",
    "x = keras.layers.Dense(units=10*Channel_uses, activation='elu', name='tx_10')(x)\n",
    "x = keras.layers.Dense(units=2*Channel_uses, activation='elu', name='tx_out')(x)\n",
    "xp = keras.layers.Lambda(perturbation, name='Xp')(x)\n",
    "concat = keras.layers.concatenate([x,xp], axis=1)\n",
    "policy = keras.layers.Lambda(Int_layer)(concat)\n",
    "print(concat.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CuMzNvRbIe62"
   },
   "source": [
    "We define the entire graph but for simplicity sake, we also define a sub-model for getting the internediate layer outputs.\n",
    "\n",
    "To be even more precise, we add perturbation after we get the Tx layer output. So, to get the perturbation matrix out, we define a full model and another proxy model (which shares weights with the full model) which return without perturbation matrix effects.\n",
    "\n",
    "We then subtract these two layers to get the value of W (perturbation matrix) for a given batch/sample\n",
    "\n",
    "(Note that we had to take this roundabout method to get W because Keras can't return two tensors for a said layer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "i-1pACOyIe63"
   },
   "outputs": [],
   "source": [
    "tx_model = keras.models.Model(inputs=tx_input, outputs=concat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 397
    },
    "colab_type": "code",
    "id": "su8nzEbBIe67",
    "outputId": "2d41ea89-c26b-4eb3-b367-fd4cc3f3503a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "tx_input (InputLayer)           (None, 1)            0                                            \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_v1 (BatchNo (None, 1)            4           tx_input[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "tx_10 (Dense)                   (None, 40)           80          batch_normalization_v1[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "tx_out (Dense)                  (None, 8)            328         tx_10[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "Xp (Lambda)                     (None, 8)            0           tx_out[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "concatenate (Concatenate)       (None, 16)           0           tx_out[0][0]                     \n",
      "                                                                 Xp[0][0]                         \n",
      "==================================================================================================\n",
      "Total params: 412\n",
      "Trainable params: 410\n",
      "Non-trainable params: 2\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "tx_model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0aVC8d9RIe6_"
   },
   "outputs": [],
   "source": [
    "pert_model = keras.models.Model(inputs=tx_input, outputs=policy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 435
    },
    "colab_type": "code",
    "id": "WGQSrJH6Ie7B",
    "outputId": "769c7504-7622-431d-e6aa-b6db943fdfb8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "tx_input (InputLayer)           (None, 1)            0                                            \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_v1 (BatchNo (None, 1)            4           tx_input[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "tx_10 (Dense)                   (None, 40)           80          batch_normalization_v1[0][0]     \n",
      "__________________________________________________________________________________________________\n",
      "tx_out (Dense)                  (None, 8)            328         tx_10[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "Xp (Lambda)                     (None, 8)            0           tx_out[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "concatenate (Concatenate)       (None, 16)           0           tx_out[0][0]                     \n",
      "                                                                 Xp[0][0]                         \n",
      "__________________________________________________________________________________________________\n",
      "lambda (Lambda)                 ()                   0           concatenate[0][0]                \n",
      "==================================================================================================\n",
      "Total params: 412\n",
      "Trainable params: 410\n",
      "Non-trainable params: 2\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "pert_model.compile(loss=tx_loss, optimizer='sgd')\n",
    "pert_model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "R8Nwp3t8Ie7E"
   },
   "source": [
    "### Rx model\n",
    "\n",
    "In the said RX model, we are taking the Perturbed input, adding channel effects and then passing on for estimation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8zflQdUdIe7E"
   },
   "outputs": [],
   "source": [
    "rx_input = keras.layers.Input((2*Channel_uses,), name='rx_input')\n",
    "# channel layer\n",
    "y = keras.layers.Lambda(lambda x: x+keras.backend.random_normal(\n",
    "        shape = (2*Channel_uses,), mean=0.0, stddev=Noise_variance**0.5), name='channel')(rx_input)\n",
    "\n",
    "y = keras.layers.Dense(2*Channel_uses, activation='relu', name='rx_2')(y)\n",
    "y = keras.layers.Dense(10*Channel_uses, activation='relu', name='rx_10')(y)\n",
    "pred = keras.layers.Dense(1, activation='relu', name='rx_output')(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 340
    },
    "colab_type": "code",
    "id": "Sg6jt5T2Ie7H",
    "outputId": "1ba164ca-33c9-4ba7-f415-a4dfe9706c2c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "rx_input (InputLayer)        (None, 8)                 0         \n",
      "_________________________________________________________________\n",
      "channel (Lambda)             (None, 8)                 0         \n",
      "_________________________________________________________________\n",
      "rx_2 (Dense)                 (None, 8)                 72        \n",
      "_________________________________________________________________\n",
      "rx_10 (Dense)                (None, 40)                360       \n",
      "_________________________________________________________________\n",
      "rx_output (Dense)            (None, 1)                 41        \n",
      "=================================================================\n",
      "Total params: 473\n",
      "Trainable params: 473\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "rx_model = keras.models.Model(inputs=rx_input, outputs=pred)\n",
    "rx_model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 94
    },
    "colab_type": "code",
    "id": "0vfLBVRrIe7J",
    "outputId": "11c30021-1212-49b5-c9b3-2d4d65da7014"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/losses_utils.py:170: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n"
     ]
    }
   ],
   "source": [
    "rx_model.compile(loss='mse', optimizer='sgd')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 55
    },
    "colab_type": "code",
    "id": "wB9cUKzoIe7L",
    "outputId": "4b665954-2fee-4b09-cd4e-bd273aab097c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1024, 16)\n",
      "(1024,) (1024,)\n"
     ]
    }
   ],
   "source": [
    "data_numbers = np.random.uniform(0,1,(Batch_size,))\n",
    "y = tx_model.predict(data_numbers)\n",
    "print(y.shape)\n",
    "XP = y[:,y.shape[-1]//2:]\n",
    "estimated_vector  = np.squeeze(rx_model.predict(XP))\n",
    "print(estimated_vector.shape, data_numbers.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Rik5r06aIe7O"
   },
   "outputs": [],
   "source": [
    "l = (estimated_vector-data_numbers)**2\n",
    "l_hat = rx_model.predict(tx_model.predict(data_numbers)[:,2*Channel_uses:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 132
    },
    "colab_type": "code",
    "id": "U0lgWzrDIe7Q",
    "outputId": "12548fd5-e023-41d7-9df1-b7e9b46289c6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n",
      "1024/1024 [==============================] - 0s 158us/sample - loss: 0.0062\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f8ea73e2c50>"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pert_model.fit(data_numbers, l_hat, batch_size=Batch_size, epochs=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zMFWoKcpIe7U"
   },
   "source": [
    "### Training\n",
    "\n",
    "Training this entire network is done as discussed in the paper -\n",
    "1. Generate a batch of numbers sampled from Uniform random variable from [0,1]\n",
    "2. Pass the numbers through Tx and then Rx\n",
    "3. Get a loss vectors for the said batch of numbers\n",
    "4. Train the Rx network on MSE with SGD\n",
    "5. Feed back the loss vector to Tx using the same pair of Tx and Rx to incorporate noise into the loss vector\n",
    "6. Use policy function, the loss vector and train the Tx for the same batch of numbers\n",
    "7. Back to step 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 47517
    },
    "colab_type": "code",
    "id": "W699EvJ4Ie7U",
    "outputId": "e9956cee-7530-41ce-f624-15ba522935e5",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1024/1024 [==============================] - 0s 129us/sample - loss: 0.3249\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.2921\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.2890\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.2707\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.2544\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.2458\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.2376\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.2120\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.2305\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1957\n",
      "1024/1024 [==============================] - 0s 4us/sample - loss: 0.1952\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1847\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1672\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1581\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1506\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1518\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1489\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1366\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1431\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1318\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1147\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1209\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1184\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1087\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1155\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1098\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1009\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.1008\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0918\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0909\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0921\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0960\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0947\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0854\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0868\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0850\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0870\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0801\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0799\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0767\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0762\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0747\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0776\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0754\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0764\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0757\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0714\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0663\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0696\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0737\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0683\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0682\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0702\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0638\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0634\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0627\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0617\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0634\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0584\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0675\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0564\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0629\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0600\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0588\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0584\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0576\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0569\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0574\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0592\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0573\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0578\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0527\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0555\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0561\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0562\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0549\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0542\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0526\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0528\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0534\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0553\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0522\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0536\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0512\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0509\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0512\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0524\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0501\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0498\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0498\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0484\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0477\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0475\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0479\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0500\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0497\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0470\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0472\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0469\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0452\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0491\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0466\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0465\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0457\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0440\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0445\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0438\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0448\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0467\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0456\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0451\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0422\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0432\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0435\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0435\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0442\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0415\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0388\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0405\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0419\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0411\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0390\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0418\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0391\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0403\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0395\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0392\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0384\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0376\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0371\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0362\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0369\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0348\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0356\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0358\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0340\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0345\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0348\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0347\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0356\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0343\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0341\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0357\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0327\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0338\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0313\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0339\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0309\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0331\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0335\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0320\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0318\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0309\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0299\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0312\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0318\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0314\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0296\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0290\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0291\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0293\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0277\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0305\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0280\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0283\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0262\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0293\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0270\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0264\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0249\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0264\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0260\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0257\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0258\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0244\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0258\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0256\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0247\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0225\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0227\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0214\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0251\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0218\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0233\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0218\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0219\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0221\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0232\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0213\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0251\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0223\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0221\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0214\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0207\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0212\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0206\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0204\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0204\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0185\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0192\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0198\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0204\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0188\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0190\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0197\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0201\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0190\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0189\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0188\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0187\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0177\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0167\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0162\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0170\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0166\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0165\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0161\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0171\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0155\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0148\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0151\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0167\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0142\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0143\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0147\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0160\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0150\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0152\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0128\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0136\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0146\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0132\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0130\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0133\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0131\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0128\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0134\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0136\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0119\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0136\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0118\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0122\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0118\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0115\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0107\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0124\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0115\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0114\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0118\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0117\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0116\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0096\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0111\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0102\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0106\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0104\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0095\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0099\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0102\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0100\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0093\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0092\n",
      "1024/1024 [==============================] - 0s 6us/sample - loss: 0.0096\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0095\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0101\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0085\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0101\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0095\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0090\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0083\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0081\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0074\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0086\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0078\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0072\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0082\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0079\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0083\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0082\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0077\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0070\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0079\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0077\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0070\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0074\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0070\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0070\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0064\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0060\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0067\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0063\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0067\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0060\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0061\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0061\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0062\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0074\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0059\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0059\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0059\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0063\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0061\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0059\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0057\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0050\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0062\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0058\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0055\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0049\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0048\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0061\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0052\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0049\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0059\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0046\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0052\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0044\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0045\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0045\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0047\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0048\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0047\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0050\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0046\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0043\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0044\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0042\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0048\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0047\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0042\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0044\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0045\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0046\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0044\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0043\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0043\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0039\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0047\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0042\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0042\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0039\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0036\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0036\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0040\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0040\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0042\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0038\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0040\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0037\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0036\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0034\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0036\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0037\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0032\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0034\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0038\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0034\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0033\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0033\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0039\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0032\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0036\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0037\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0035\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0033\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0036\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0032\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0033\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0033\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0031\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0032\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0031\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0030\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0031\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0031\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0028\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0027\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0029\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0026\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0025\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0024\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 4us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0021\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0022\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0020\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0023\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0019\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0018\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0017\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0016\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0015\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9646e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 4us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9736e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7612e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 4us/sample - loss: 0.0014\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9579e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5775e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5574e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8330e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8264e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7873e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9268e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8467e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4424e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0013\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 9.6966e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9822e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6062e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0195e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4164e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7374e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8728e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8077e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0646e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7680e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7950e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2228e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6106e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5878e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8896e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5506e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9238e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6250e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8618e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5650e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6078e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4617e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5031e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7954e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8597e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1897e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6853e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8603e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6681e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3341e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7244e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1004e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5724e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2709e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4065e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9221e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6848e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9642e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3439e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5824e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5016e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8674e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5156e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9785e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9679e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7985e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5814e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4337e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3887e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3778e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1757e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9412e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3307e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1594e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0075e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5733e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4127e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4940e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2027e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3963e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4163e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1406e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8030e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2945e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 9.5572e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0576e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5332e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2719e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9641e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8434e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1721e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9562e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0228e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9328e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0668e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 9.5405e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7467e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1361e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5490e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3058e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9564e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.6099e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3435e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1930e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8436e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.6193e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.7914e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1615e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9123e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7376e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2529e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.8063e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1434e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 9.3875e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3748e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1349e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4874e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7962e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0011\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6125e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5560e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2682e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9979e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3346e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8701e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9235e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5081e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4392e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2796e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2443e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1481e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0374e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5084e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3920e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3418e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1706e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4049e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7781e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9115e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9573e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5149e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0214e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2499e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0010\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7213e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4609e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2134e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1950e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8389e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4196e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9483e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.6636e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0100e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4949e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1309e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4559e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3972e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7751e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4767e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6388e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8090e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3307e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4919e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4738e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8655e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2069e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4349e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2585e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1849e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7523e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8349e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.6405e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3343e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0221e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2465e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0790e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9752e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1198e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3876e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8498e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8219e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8922e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3183e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4615e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6467e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.6645e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2566e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2253e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0530e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0969e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 7.3113e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3873e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4692e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1669e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4303e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7974e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 8.4505e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2815e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1233e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1998e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8457e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4086e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3750e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1048e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4389e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1956e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0459e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1486e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8216e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8156e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3170e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 8.4065e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1560e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1618e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4165e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9211e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6475e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7020e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1785e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4415e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2749e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0060e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3580e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7972e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4512e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2719e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5168e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0583e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7134e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4650e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6246e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.7911e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4275e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2125e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4013e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5058e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8698e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3830e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6802e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.1035e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2720e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8075e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4202e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.3358e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9428e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7735e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2294e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1716e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1346e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8821e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4802e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5559e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0337e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7565e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3590e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4598e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8070e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.9103e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0470e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3406e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.8759e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8587e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4918e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8919e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7299e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3248e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2177e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2637e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4299e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1820e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0545e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9913e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2572e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.1846e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 0.0012\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9426e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3211e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.0237e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1748e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3864e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.4917e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8130e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3258e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4584e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1790e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3592e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8260e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0741e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7741e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7143e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3744e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3492e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3078e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0789e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8124e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.8370e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.2297e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6487e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2871e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1358e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5280e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2582e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5452e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5976e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 7.2931e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9023e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1851e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1293e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9533e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1952e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0468e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7784e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3546e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0909e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2489e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0674e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4817e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5945e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7579e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1294e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9096e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3496e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6678e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7445e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4109e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7070e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8373e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6600e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3805e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6419e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3935e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5817e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0122e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7502e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9429e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3396e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9677e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2903e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9182e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7042e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4597e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9137e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3064e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3098e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2270e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9543e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.9978e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0929e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6922e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7194e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5485e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9084e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6904e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0272e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2242e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0896e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7114e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2302e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7908e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7734e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5967e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3083e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7766e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3423e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1501e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7031e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9639e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7058e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0964e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0542e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3493e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9935e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.5351e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3319e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9227e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5477e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2053e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5247e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6882e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3251e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6364e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9481e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5503e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2014e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5123e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6981e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0595e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3318e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4548e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1477e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5587e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4764e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1250e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6472e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5277e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 6.8074e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2072e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2621e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4763e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7299e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6370e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7883e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2984e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7754e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3143e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5125e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7972e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0597e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3596e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9782e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0321e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 9.0781e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4024e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7473e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4783e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2190e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7130e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4602e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 6.2641e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5735e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3213e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.4990e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.6869e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9029e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2674e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9973e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1605e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3946e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6536e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6641e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4559e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5169e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6592e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7121e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0701e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0980e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7008e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0364e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2928e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4437e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3880e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5771e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1522e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3084e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0886e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9877e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1921e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6753e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2265e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0779e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0703e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8642e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3783e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9117e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3555e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5563e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3477e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2395e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8918e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6360e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7690e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1906e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0289e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8070e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1017e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0131e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1392e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9219e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8266e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2611e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0713e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9724e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7885e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.5383e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0970e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6863e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0214e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0821e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0628e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8676e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5561e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4068e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9036e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8474e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9991e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0099e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0187e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8244e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2348e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1492e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1449e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.4368e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0538e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6827e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3237e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3617e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2631e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3257e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8001e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0093e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6602e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.9142e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8690e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1489e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4540e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4770e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.3753e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7078e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3784e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.7801e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9169e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9901e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3040e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4496e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5885e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4517e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6351e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0039e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3149e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3935e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1577e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8640e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3283e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.5846e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4429e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 6.0559e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.8985e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2665e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2525e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7182e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4277e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0376e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3928e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8623e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9439e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7453e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5739e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9751e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7294e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5640e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5588e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3867e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9628e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0948e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9684e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4788e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4019e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0240e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7565e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1101e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 8.2820e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1877e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4068e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1731e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2941e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5818e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7473e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 5.9520e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3043e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3199e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0286e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9616e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9564e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5713e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6762e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5457e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3508e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0655e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1335e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8830e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.7881e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9384e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5737e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0585e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7939e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4297e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9753e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0052e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0196e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9763e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3290e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1868e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1529e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3682e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4421e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.3840e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0486e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7181e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6713e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8129e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7193e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5539e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 5.7506e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6538e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0975e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1673e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7551e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1758e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 5.9567e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0168e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2173e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2918e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8298e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2022e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8501e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0410e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3123e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8770e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1020e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9299e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8258e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7871e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2927e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0700e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4829e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7376e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8115e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0198e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0005e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8562e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8007e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5995e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3251e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0102e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4353e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2582e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7149e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7064e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4912e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7990e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2177e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2458e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9665e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6076e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6892e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7792e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1386e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4450e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3221e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9288e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6784e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4022e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2505e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5908e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9963e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5520e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.2599e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4346e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4824e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3637e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0752e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9900e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4460e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8799e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7894e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3707e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0537e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1252e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2544e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6806e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0552e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8811e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6097e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1051e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5169e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3560e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5750e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2063e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2919e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9847e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4759e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1260e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0674e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7059e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4135e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6527e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3290e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6075e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.8822e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8382e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5774e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9550e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4659e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2393e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9989e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3116e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2570e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1011e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0653e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3464e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9672e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7156e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0931e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1614e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9457e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9094e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6911e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6997e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3462e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2827e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8226e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9630e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2684e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7010e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9681e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2968e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.9050e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4890e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7234e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8836e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7695e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8431e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4007e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1796e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1146e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9873e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9368e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0774e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2354e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2325e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7645e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1385e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6899e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4430e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.0245e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8436e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8158e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8860e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5455e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9122e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9223e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0797e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8200e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4586e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8582e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9612e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2789e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0590e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1528e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4414e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2740e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7190e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7337e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0749e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6215e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7565e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4629e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9921e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.2568e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2553e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3523e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0576e-04\n",
      "1024/1024 [==============================] - 0s 7us/sample - loss: 5.0710e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5690e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9452e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4996e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1141e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5614e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8678e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3989e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6095e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6433e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3853e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0208e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8152e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3232e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.1724e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3279e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6606e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.5275e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4433e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0331e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8211e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8830e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8334e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1672e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0306e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0044e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8438e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4698e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5923e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2319e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4413e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6274e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5240e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6658e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3336e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9565e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8487e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8544e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1405e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6030e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8482e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1947e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0874e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.9313e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.7853e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1932e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4236e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3520e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7939e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6523e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8905e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8506e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.5958e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4656e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0803e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1378e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5793e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4194e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3560e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6434e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6162e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3307e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6916e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9153e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8747e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3166e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9436e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0072e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6666e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5118e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0076e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8536e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5279e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7451e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3336e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9760e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8424e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0215e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7957e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9624e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7593e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9204e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8561e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7109e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7193e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9212e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3846e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6595e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7048e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5593e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 7.3527e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4491e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0317e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6289e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7641e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8750e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1831e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4211e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6887e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8445e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1176e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6189e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8582e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0049e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1559e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2137e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1304e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8461e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7128e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0467e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1298e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7329e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5910e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8516e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5006e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8881e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9789e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7910e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4396e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3223e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6274e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2925e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6326e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3486e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3467e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7788e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.6327e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1220e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5333e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2092e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7426e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2926e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8872e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3585e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4526e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2683e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0443e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5861e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6202e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6532e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5434e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1828e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8804e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0026e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3127e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4571e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7437e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5068e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9780e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8154e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9906e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8801e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7649e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8240e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2884e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3984e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5323e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7492e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5285e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2396e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2033e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2572e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2564e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0020e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8576e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3904e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7867e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9237e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0928e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7879e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4214e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4728e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2846e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0369e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5412e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2177e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2092e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3402e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5943e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3771e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4414e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9491e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3815e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5567e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4462e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2581e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4308e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7844e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9774e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6816e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7955e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3314e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3998e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5055e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8241e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3162e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0019e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2508e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0377e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1029e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0440e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2956e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0925e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9908e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0128e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2651e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9800e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0634e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6787e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5422e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2477e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3912e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1010e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2635e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3078e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3680e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2814e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6018e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1391e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6252e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0273e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8742e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0586e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0522e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6241e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3076e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1364e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3219e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8158e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0252e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2539e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8672e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5382e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2059e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1720e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8398e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9784e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4951e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1604e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0806e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8819e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7695e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4946e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6322e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3611e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2405e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1812e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6554e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4056e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8847e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7305e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2033e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5343e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2354e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3766e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4567e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8726e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0274e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9307e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9117e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0167e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9981e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3591e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7170e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 3.7937e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0242e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0524e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9766e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1218e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.0174e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1005e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.2217e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4190e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0127e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6963e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3281e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3883e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4642e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7724e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8240e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1273e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2169e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2191e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8270e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1136e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0657e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9129e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6785e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.8488e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8430e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3006e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6456e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8385e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5830e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9716e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6742e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9754e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8823e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0449e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2200e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7910e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6680e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6149e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9554e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1541e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7420e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5252e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9896e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9672e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7654e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1644e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0814e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6772e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0524e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1167e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8813e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8606e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8145e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8324e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1092e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5329e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5568e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9562e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0410e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.4796e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.9543e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.6987e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9083e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7475e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3379e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9839e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8067e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1029e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8392e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2064e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7698e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2889e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5565e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8474e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6667e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5244e-04\n",
      "1024/1024 [==============================] - 0s 4us/sample - loss: 3.7524e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7775e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0515e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 3.7194e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9872e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6735e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.0890e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5652e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6767e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2136e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4061e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5683e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2644e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 3.5724e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7913e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5455e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1827e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.4298e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9522e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4777e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2679e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5953e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7248e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8241e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3785e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4734e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1163e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5022e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9268e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6231e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6727e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5255e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0172e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8827e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0110e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1958e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8769e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0404e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4251e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4409e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2650e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9574e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4117e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6132e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3528e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7617e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0746e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6065e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6355e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5770e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7466e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8601e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3802e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7292e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 3.7947e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2115e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0169e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4860e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8522e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5884e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7253e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0736e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8025e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5511e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6256e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9778e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2378e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5373e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7312e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6202e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3468e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8881e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7009e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8775e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8085e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0423e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3868e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9926e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7134e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6877e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8042e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2551e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7679e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6253e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0181e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.2174e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1352e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8910e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7902e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4096e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0403e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4261e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1338e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5656e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1203e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3977e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4526e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0256e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6533e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4613e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6056e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.5538e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9546e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3485e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.1024e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6343e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.4009e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4063e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 5.0581e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6164e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9424e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.5644e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6528e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 3.8475e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7367e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0446e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3279e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8757e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8653e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6148e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3813e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3291e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8770e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0648e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6008e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9127e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.1275e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 6.1413e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9128e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9206e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8797e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4252e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2214e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9052e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.6642e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4968e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8358e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.0116e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2337e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7976e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4285e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4585e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7099e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6567e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7121e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5841e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6947e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7485e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.1981e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8457e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7602e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9438e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9601e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5157e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1608e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2884e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5656e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.0951e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8229e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8094e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 3.3549e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9548e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4513e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9285e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6949e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9972e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8997e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9482e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7872e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6228e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3293e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6649e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.0845e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7604e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6478e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8008e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3366e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4062e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.7309e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0268e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.6065e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3244e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.0335e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.6164e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5558e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.0696e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9933e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4189e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6255e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4742e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5893e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8889e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4687e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6309e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4329e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4726e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3333e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9076e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8307e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.0444e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8222e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.7546e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7734e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4465e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6325e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9550e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.1333e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8273e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7096e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7516e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.1782e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9795e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4575e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9280e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3065e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8834e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3407e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4083e-04\n",
      "1024/1024 [==============================] - 0s 4us/sample - loss: 3.5521e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5229e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9609e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1402e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5364e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6065e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6734e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5097e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8865e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.9562e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4095e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4309e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6919e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3726e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7751e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2768e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5333e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3808e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2108e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3834e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4556e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9498e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3980e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2909e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.1069e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.9166e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2770e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5343e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2767e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.8274e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 4.3953e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.4243e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2263e-04\n",
      "1024/1024 [==============================] - 0s 3us/sample - loss: 3.8198e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8142e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8160e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5561e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 5.3372e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4638e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.2437e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.7643e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3121e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.8363e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.6473e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4674e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 4.3939e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3245e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3640e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.3033e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.4603e-04\n",
      "1024/1024 [==============================] - 0s 2us/sample - loss: 3.5091e-04\n"
     ]
    }
   ],
   "source": [
    "for i in range(Epochs):\n",
    "    data_numbers = np.random.uniform(0,1,(Batch_size,))\n",
    "    y = tx_model.predict(data_numbers)\n",
    "    XP = y[:,y.shape[-1]//2:]\n",
    "    estimated_vector= np.squeeze(rx_model.predict(XP))\n",
    "    l = (estimated_vector-data_numbers)**2\n",
    "    l_hat = rx_model.predict(tx_model.predict(data_numbers)[:,2*Channel_uses:])\n",
    "    pert_model.fit(data_numbers, l_hat, batch_size=Batch_size, epochs=1, verbose=0)\n",
    "#     print(\"Tx-done\")\n",
    "    rx_model.fit(XP, data_numbers, batch_size=Batch_size, epochs=1)\n",
    "#     print(\"Rx-done\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WWLkEPpTIe7Y"
   },
   "source": [
    "### Prediction phase\n",
    "Note that the network is predicting numbers with a quite low error margin (+- 1e-2)\n",
    "This is in case of continous numbers \n",
    "Say we feed numbers sampled from PAM (discrete numbers) and set our prediction rules as a floor or ceiling function, this model easily achieves 95% accuracy\n",
    "\n",
    "This is all achieved even though there is a noisy feedback of losses from Tx to Rx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 93
    },
    "colab_type": "code",
    "id": "NKcUoNGJIe7Z",
    "outputId": "5b0551d0-9615-4690-bf1b-3c35248ab5c7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.45243852 0.77047394 0.05366295 0.12279146 0.44764297 0.47919261\n",
      " 0.00117869 0.9767389  0.0882423  0.37715078]\n",
      "[0.43628636 0.79524827 0.08269244 0.12927    0.43189043 0.4759401\n",
      " 0.05670264 0.9522356  0.10295658 0.38193107]\n"
     ]
    }
   ],
   "source": [
    "data_numbers = np.random.uniform(0,1,(10,))\n",
    "y = tx_model.predict(data_numbers)\n",
    "XP = y[:,y.shape[-1]//2:]\n",
    "estimated_vector= np.squeeze(rx_model.predict(XP))\n",
    "print(data_numbers)\n",
    "print(estimated_vector)\n",
    "# l = (estimated_vector-data_numbers)**2\n",
    "# l_hat = rx_model.predict(tx_model.predict(data_numbers)[:,2*Channel_uses:])\n",
    "# pert_model.fit(data_numbers, l_hat, batch_size=Batch_size, epochs=1)\n",
    "# print(\"Tx-done\")\n",
    "# rx_model.fit(XP, data_numbers, batch_size=Batch_size, epochs=1)\n",
    "# print(\"Rx-done\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bqHp3kEEIe7e"
   },
   "source": [
    "### Post implimentation tid-bits\n",
    "\n",
    "Please note that we had to make some chnages from the original discussed implimentation and theory to attain some numerical stability and to dodge NaN losses\n",
    "\n",
    "1. In the J(w,$\\theta$) function, we have a part involving exp(|w|) and some constants. Where as the loss involved $L_i$ * log(J(w,$\\theta$)). This causes numerical instability in case the J function goes negative or is very very small due to exp() and then log. To prevent this, we ignored the constants (as they dont affect gradient terms while differentiating) and removed the exp() and log() terms all-together\n",
    "\n",
    "2. Author assumed two pairs of Tx-Rx with shared weights. We used one for both purposes as it is symmetric"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4IvlRDqDRxQy"
   },
   "source": [
    "### Modulation example\n",
    "\n",
    "As we mentioned before, here is an 8 PAM example. We generate real numbers between 0 and 1 for 8 samples and pass through the network and decode accordingly and check accuracy. Since this is a fairly small example, it attains 100% accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 93
    },
    "colab_type": "code",
    "id": "TKaQT0GENza-",
    "outputId": "8c0e1238-7257-4b97-9bae-151ee9bc32f0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.         0.14285714 0.28571429 0.42857143 0.57142857 0.71428571\n",
      " 0.85714286 1.        ]\n",
      "[0 1 2 3 4 5 6 7]\n",
      "[0.07142857142857142, 0.21428571428571427, 0.3571428571428571, 0.5, 0.6428571428571428, 0.7857142857142856, 0.9285714285714286]\n"
     ]
    }
   ],
   "source": [
    "# Generate 8-PAM data\n",
    "modulation_data = np.linspace(0,1,8)\n",
    "modulation_inds = np.linspace(0,7,8)\n",
    "print(modulation_data)\n",
    "modulation_inds = modulation_inds.astype(int)\n",
    "print(modulation_inds)\n",
    "decision_regions = []\n",
    "for i in range(len(modulation_data) -1):\n",
    "  decision_regions.append((modulation_data[i]+modulation_data[i+1])/2.0)\n",
    "\n",
    "print(decision_regions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 112
    },
    "colab_type": "code",
    "id": "eBNm5ku3PKBg",
    "outputId": "d748cd60-e05b-49b6-916e-563e865ae340"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.         0.14285714 0.28571429 0.42857143 0.57142857 0.71428571\n",
      " 0.85714286 1.        ]\n",
      "[0.05598246 0.13984698 0.2771691  0.41218558 0.5818432  0.73225975\n",
      " 0.8532618  0.95760846]\n",
      "[0 1 2 3 4 5 6 7]\n"
     ]
    }
   ],
   "source": [
    "y = tx_model.predict(modulation_data)\n",
    "XP = y[:,y.shape[-1]//2:]\n",
    "estimated_vector= np.squeeze(rx_model.predict(XP))\n",
    "print(modulation_data)\n",
    "print(estimated_vector)\n",
    "\n",
    "est_sig = []\n",
    "for i in estimated_vector:\n",
    "  index = (np.abs(modulation_data-i)).argmin()\n",
    "  est_sig.append(modulation_inds[index])\n",
    "  \n",
    "est_sig = np.array(est_sig)\n",
    "print(est_sig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 36
    },
    "colab_type": "code",
    "id": "G652PMATQ7cy",
    "outputId": "576161b3-6c79-456c-d69f-ae4faf1797d1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "print(accuracy_score(est_sig, modulation_inds))"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "code.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
